Saliency-Based Apparatus And Methods For Visual Prostheses

ABSTRACT

Saliency-based apparatus and methods for visual prostheses are disclosed. A saliency-based component processes video data output by a digital signal processor before the video data are input to the retinal stimulator. In a saliency-based method, an intensity stream is extracted from an input image, feature maps based on the intensity stream are developed, plural most salient regions of the input image are detected and one of the regions is selected as a highest saliency region.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a divisional application of U.S. patentapplication Ser. No. 12/043,830, filed Mar. 6, 2008, for Saliency-BasedApparatus and Method for Visual Prosthesis, which claims the benefit ofpriority to U.S. Provisional Patent Application Ser. No. 60/905,643filed on Mar. 8, 2007, the contents of which are incorporated herein byreference in their entirety.

STATEMENT OF GOVERNMENT GRANT

At least part of the present disclosure is based on work supported bythe National Science Foundation under Grant No. EEC-0310723. At leastpart of the present disclosure was also made with governmental supportunder grant No. R24EY12893-01, awarded by the National Institutes ofHealth. The U.S. Government may have certain rights in the presentinvention.

FIELD

The present disclosure relates to visual prostheses. In particular, itrelates to saliency-based apparatus and methods for visual prostheses.

BACKGROUND

Electronic retinal prostheses for treating retinal degenerative diseasessuch as retinitis pigmentosa (RP) and age-related macular degeneration(AMD) are known. In these diseases, the photoreceptor cells of theretina are affected but other retinal cells remain relatively intact.Hence, the retinal prosthesis electrically activates these remainingcells of the retina to create artificial vision. External components areused to acquire and code image data for transmission to an implantedretinal stimulator.

FIG. 1 shows a schematic representation of the main components of avisual prosthesis taken from U.S. published patent application2005/0288735, incorporated herein by reference in its entirety. Inparticular, the external portion (1) of the visual prosthesis comprisesan imager (10), e.g., a video camera to capture video in real time, anda video data processing unit (20) comprising a Digital Signal Processor(DSP) to process video data and then send output commands (30) for aretinal stimulator to be implanted on the retina of a patient.Currently, the video camera (10) outputs images having resolution ofabout 640×480 to the DSP in real time. The DSP takes in these images anddoes image processing on each of these input frames. The processing ratecan be of about 3 to 30 image frames per second depending on theprocessing method or algorithm. The output (30) of the DSP is to be sentto the telemetry block (3) of the retinal prosthesis system, which willnot be explained here in detail.

The electrode grid of the retinal stimulator (40) to be implanted in theeye is currently of size 10×6. This implies that the resolution of theimage output (30) from the DSP contained in the video data processingunit (20) is to be 10×6. A 10×6 size electrode grid array significantlyreduces the field of vision that can be offered to a patient. Currently,such field of view is about 20 degrees. With a restricted field ofvision, the patient has to scan the scene in front with head movementsand find out the region that looks significant or worth giving attentionto.

Saliency methods or algorithms are known in the art. Generally speaking,a saliency method treats different parts of the same image differently,focusing only on relevant portions or sections of that image. A saliencyalgorithm per se is known from Itti et al (see “A model foraliency-based search visual attention for rapid scene analysis” byLaurent Itti, Christof Koch and Ernst Niebur, IEEE Transactions forPattern Analysis and Machine Intelligence, Vol 20, No 11 November 1998and “A saliency-based search mechanism for overt and covert shifts ofvisual attention” by Laurent Itti and Kristof Koch, both of which areincorporated herein by reference in their entirety). In particular, Ittiet al use a visual-attention model which is based on an early primatevisual attention model and aims at finding the most salient region of animage frame at any given instant. According to Itti et al's algorithm,saliency at a given location is determined primarily by how differentthe location is from its surround in color, orientation, motion, depthand so on.

FIG. 2 shows a general overview of Itti et al's saliency algorithm. Inparticular, the algorithm concentrates on the color (50), intensity (60)and orientation (70) information in any given image frame to calculatethe most conspicuous location of an input image. 7 streams ofinformation like intensity, Red-Green color, Blue-Yellow color andorientations at 0, 45, 90 and 135 degrees are extracted from the inputimage. Image pyramids with 9 levels are constructed for each of theinformation streams. Feature-maps for the center-surround structure aredeveloped by taking the difference between the finer and coarser scalesof the pyramids. Three conspicuity maps are constructed by summing thefeature-maps of the three kinds of information after normalization.Normalization is a process to combine different modalities likeintensity, color and orientation. Normalization helps to promote themaps having strong peaks and suppress the maps having comparable peaksand to bring feature-maps of the different information streams at thesame level in order to linearly sum them to form the final saliency map.The saliency conspicuity map is constructed by summing the conspicuitymaps for color, intensity and orientation, thus obtaining atopographically arranged map that represents visual saliency of theinput image. A region surrounding the pixel with the highest gray-scalevalue is taken as the most salient region.

SUMMARY

According to a first aspect, a visual prosthesis is provided,comprising: an image processor to process video data from an imager; avisual stimulator to stimulate visual neurons on a patient; and asaliency-based component to process the video data before the video dataare input to the visual stimulator, the saliency-based component beingconfigured to apply a saliency algorithm on the video data.

According to a second aspect, a method of providing video data to asubject with an implanted visual stimulator is provided, the implantedvisual stimulator associated to an image processor processing video datafrom an input image, comprising: extracting only an intensity streamfrom the input image; developing feature maps based on the intensitystream; detecting a plurality of most salient regions of the input imagebased on the feature maps; selecting one salient region of the mostsalient regions as a highest saliency region; and providing highestsaliency region information to the subject

According to a third aspect, an apparatus for providing stimulationsignals to a visual stimulator is provided, comprising: a saliency mapgenerator to process a video input coming from a camera image; adownsampling component to downsample saliency map resolution to visualstimulation resolution; and a stimulation component to providestimulation signals based on the downsampled saliency map resolution.

According to a fourth aspect, an apparatus for providing stimulationsignals to a visual stimulator is provided, comprising: a saliency mapgenerator to process a video input coming from a camera image andgenerate a saliency map; and a stimulation component to providestimulation signals to the visual stimulator based on the saliency mapgenerated by the saliency map generator.

According to a fifth aspect, a method for providing stimulation signalsto a visual stimulator is provided, comprising: generating a saliencymap of an input image; separating the input image into multiple regionsaccording to saliency values of the saliency map; and separatingstimulation of the multiple regions through the visual stimulator intime in accordance with a rastering process.

According to a sixth aspect, a method of providing video data to asubject with an implanted visual stimulator is provided, the implantedvisual stimulator associated to a processor processing video data froman input image, the method comprising: detecting salient regions of theinput image through a discrete wavelet transform process.

According to a seventh aspect, an apparatus for providing video data toa subject with an implanted visual stimulator is provided, the implantedvisual stimulator associated to a processor processing video data froman input image, the apparatus comprising: a module configured to detectsalient regions of the input image through a discrete wavelet transformprocess.

According to an eighth aspect, an apparatus for providing video data toa subject with an implanted visual stimulator is provided, the implantedvisual stimulator associated to a processor processing video data froman input image, the apparatus comprising: a module configured to detectsalient regions of the input image through a discrete wavelet transformprocess operating on two-dimensional data, the module comprising aplurality of levels of filters, each level comprising a first sub-leveloperating on data in a first dimension and a second sub-level operatingon data in the second dimension.

According to a ninth aspect, a method of providing video data to asubject, comprising: receiving video data having a first resolution;processing the video data in accordance to a saliency method to obtainprocessed video data having a second resolution, lower than the firstresolution, the processing being adapted to prefer peripherally locatedvideo data to centrally located video data once the peripherally locatedvideo data is more salient than the centrally located video data; andsending the processed video data to a visual stimulator implanted on thesubject.

According to a tenth aspect, a visual prosthesis is provided,comprising: a camera sensitive to ultraviolet light; a visual stimulatorto stimulate visual neurons on a patient; and means to process videodata output from the camera, the video data including data correspondingto the ultraviolet light, before the video data are input to the visualstimulator.

Further aspects and embodiments of the present disclosure will becomeclear to the person skilled in the art upon reading of thespecification, drawings and claims of the present application.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic representation of the main components of avisual prosthesis.

FIG. 2 shows a general overview of Itti et al's saliency algorithm.

FIG. 3 shows a video data processing unit in accordance with anembodiment of the present disclosure.

FIG. 4 shows some steps of a saliency method according to an embodimentthe present application.

FIGS. 5-7 shows schematic diagrams of saliency map-based stimulationapproached.

FIGS. 8 and 9 shows saliency methods based on a discrete wavelettransform.

DETAILED DESCRIPTION

The present disclosure applies a “saliency” method or algorithm to avisual prosthesis. In particular, the saliency algorithm, method orblock will further process the video data processed by the DSP beforethey are input to the 10×6 electrode grid. For example, the saliencyblock can operate in parallel with the DSP. Alternatively, or inaddition to that, the saliency block or module can be inside the DSP,and/or operate before or after the DSP. According to furtherembodiments, the saliency block can also operate on an input imagetogether with a (field-programmable-gate-array) FPGA or with anapplication-specific integrated circuit (ASIC). In the next paragraphs,an exemplary embodiment where the saliency block is provided after a DSPwill discussed. However, the person skilled in the art will understandthat other embodiments are possible, also on the basis of what stated inthe present paragraph.

According to an embodiment of the present disclosure, a saliencyalgorithm or method is applied to a visual prosthesis system like theone shown in FIG. 1 above. By way of example and not of limitation, FIG.3 shows a video data processing unit (200) comprising a DSP (210) and asaliency module (220). As soon as the DSP (210) processes the video datacoming from the imager (10), the saliency module (220) further processesthe output of the DSP (210), in accordance with Itti et al's algorithmor the methods later explained in the present application. The personskilled in the art will also understand that while FIG. 3 shows asaliency module (220) right after the DSP (210), other embodiments arepossible so long as the saliency module (220) is placed after the DSP(210) and before the retinal stimulator (40). More generally, theretinal stimulator (40) can be any kind of visual stimulator, forexample a stimulator stimulating the cortical regions of a patient. Asto the DSP (210), one possible choice is the TMS320 DM642 video andimaging processor from Texas Instruments. Such embodiment is well suitedfor testing purposes. However, for practical use, DSPs of small size andrunning on low power may be better suited to be used in the presentdisclosure. In other words, the digital signal processor (210) shouldrun on a limited amount of power compliant with power limits of devicesoperating on visual prostheses. Moreover, as also explained above, FPGAsand/or ASICs can be provided instead or together with DSPs.

Applicants have implemented Itti et al's saliency algorithm shown inFIG. 2 above to allow regions of interest to be detected in theperipheral visual field of a subject, thus giving audio or video cues tothe subject to look in the direction of the region of interest. An imagelocation is considered “salient” if it contains different luminance,color, texture, differently oriented structure, motion or other featuresthan the surrounding regions. Salient locations should be emphasized inthe visual presentation on the visual prosthesis device stimulation, sothey can be easily picked up by the subject. On the basis of salientregions, the patients divert their gaze in the salient region instead ofscanning an entire scene. Therefore, in accordance with the presentdisclosure, saliency is used to aid patients search and look forimportant objects in their environment and to simulate a scanningenvironment, thus achieving a good agreement between the gaze points ofa subject (which imply those points are salient for the subject in agiven image) and the salient points detected by the algorithm.

The applicants have noted that application of the above algorithm withrespect to a retinal prosthesis recipient lies in the detection ofobjects or obstacles. Some of the steps performed by applicants areshown in FIG. 4.

In particular, with this application in mind, the applicants have setthe processing rate of the algorithm on the DSP to meet about 3-5 framesper second. The applicants have also noted that Itti et al's algorithmwith 7 streams of information from the intensity, color and orientationof an image is computationally very expensive for the DSP to run it at3-5 frames per second. However, the applicants have noted that theintensity information is essentially contrast information and iscontained in just 1 out of the 7 streams being processed by the saliencyalgorithm, i.e. the intensity stream. Considering this, the applicantshave selected to use only the intensity stream for saliency detection,as shown in step S1 of FIG. 4.

Moreover, in order to detect the most salient region, instead of takingthe highest intensity pixel and a region around it as in the algorithmof FIG. 2, applicants first detect 3 most salient regions, as shown instep S5 of FIG. 4. The three regions can be detected by rank-orderingthe regions in the image (as done in Itti et al, for example) and thenby selecting the three highest-ranked regions. These regions undergosimple averaging (step S6) and then the highest intensity region isselected as the most salient region (step S7). This is done in order toavoid extremely small regions with a few high grayscale pixels to bedetected over larger regions with more pixels but slightly lessergrayscale values. This is also done so that smaller but brighter regionsin an image do not overshadow larger but slightly less intense regionsas these could potentially be the more relevant regions to a subject.

There are two types of normalization processes proposed by Itti et al,which are iterative normalization and max normalization. Normalizationis one of the most expensive routines in the Itti et al algorithm Forthis reason, applicants have opted, in one embodiment of the presentdisclosure, to not perform any kind of normalization. Such disadvantageis overcome by the fact that, in accordance with what explained above, 3most salient regions are detected and the best of them is chosen. Notperforming normalization allows a higher amount of frames per second tobe processed, as shown in the following table, where the time isexpressed in seconds and applicant's method is run on a DM642 720 MHzDSP.

Iterative Max No Normalization Normalization Normalization Gaussian0.1262 0.1276 0.1276 Pyramids Center- 0.0273 0.0248 0.0249 surround mapsNormalization 0.9919 0.0579 0.0008 at different 0.1261 0.0209 levels ofthe 0.0163 0.0108 pyramids Saliency 0.0141 0.0208 0.0175 marker Entire2.5237 0.5092 0.1955 algorithm Frames/sec 0.3962 1.9639 5.1151

The saliency method described above has been implemented on MATLAB®software. Simulink® software from Mathworks, Inc. has been used totarget the DSP processor and to load and run the code to and from theDSP.

According to a further embodiment of the present disclosure, a method ofcomputing and presenting a saliency map to a subject is described. Thesaliency map can be obtained through the basic Itti et al's algorithm,or other methods, like the one shown in FIGS. 3 and 4 above.

According to Itti et al's algorithm, as already explained above, imagefeatures such as luminance, color, and orientation are computed from ascene. A center-surround and normalization process computes the featuremaps based on the filter results. Feature maps are integrated intoconspicuity maps based on the feature categories. A linear processintegrates these into an explicit saliency map. The saliency map is thestarting point of the method according to this further embodiment.

According to a first aspect of this further embodiment, the computedsaliency map is sub-sampled and presented directly to a subject byencoding high saliency locations with brighter stimulation (or viceversa). As shown in FIG. 5, a saliency map (90) generated afterprocessing a video input coming from a camera image (80) is down-sampledto the resolution of the electrode array (e.g., a 10×6 electrode array)and presented to the subject by way of stimulation (100). Incidentally,in all of the embodiments of the present disclosure, the camera can alsobe sensitive to infrared (IR), ultraviolet (UV), ultrasound and X-rayspectra. In particular, the inventors have found that using heat (IR) asa saliency factor is beneficial. By way of example, anything that isdangerously hot is highly salient. Farher, people and most animals willshow up in IR and are salient. Moreover, visual prostheses cameras assuch, sensitive to UV light, are also provided.

According to a second aspect of this further embodiment, the computedsaliency map is used to modulate the stimulation, as shown in FIG. 6.For example, image contrast in high saliency areas can be enhanced,while low saliency image areas can be smoothed. Such aspect can becombined with the use of one or more image fusion algorithms (110). Forexample, using wavelet fusion, an image can be decomposed into waveletdomain. The wavelet coefficient, then can be multiplied or scaled usingthe corresponding saliency value. The reconstruction from such modulatedcoefficients will have higher contrast at the higher saliency areas andwill be sub-sampled to the resolution of the electrode array andpresented to the subject by way of stimulation (120).

According to a third aspect of this further embodiment, the computedsaliency map can be used to determine the stimulation sequence andassignment in a rastering scheme. In a rastering stimulation pattern,each group of electrodes is assigned to a unique stimulation temporalpattern. When presented to a new image, electrodes that correspond tohigher saliency regions can be stimulated earlier and separately in therastering pattern. Thus, when these electrodes are stimulated, therewill be more contrast since other electrodes are not stimulated. On theother hand, for the low saliency regions, the corresponding electrodeswill be stimulated later. Higher saliency means higher probability ofbeing stimulated alone, which helps to promote more contrast as shown,for example, in FIG. 7 which depicts an exemplary presentation strategyaccording to this third aspect. With reference to such figure, the inputcamera image (80) is separated into multiple regions (130)-(180)according to the saliency value. A rastering assignment module (190)separates the stimulation of regions (130)-(180) in time. Low saliencyregions (170), (180) are grouped together (310), while high saliencyregions (130), (140) and medium saliency regions (150), (160) arestimulated independently of each other (320)-(350). Moreover, accordingto this third aspect, certain regions of the image can be chosen not tobe stimulated to the subject if they fall below a certain saliencythreshold. Since not all electrodes will be stimulated, this approachhelps lowering the power consumption.

According to a yet further embodiment of the present application, asaliency algorithm or method alternative to the one devised by Itti etal and alternative to the one shown in FIGS. 3 and 4 of the presentapplication is shown. According to this method, a discrete wavelettransform (DWT) is used to extract information about size and locationof visual features such as points and edges in the image. The DWT is awavelet transform for which the wavelets are discretely sampled. Awavelet is a mathematical function used to divide a given function orsignal into different frequency components and study each component witha resolution that matches its scale.

As shown in FIG. 8, the DWT of an input signal (360) is calculated bypassing it through a series of filters. The samples are simultaneouslypassed to a first level lowpass filter (370) and a first level highpassfilter (380). After filtering, the filter outputs are downsampled bytwo. The output of the first level highpass filter (380) will providefirst level coefficients, while the output of the first level lowpassfilter (370) is input to second level lowpass (390) and highpass (400)filters, and the process is repeated.

In accordance with such embodiment, the DWT allows extraction ofinformation about the size and location of visual features such aspoints and edges in the image. The image data (360) is input as an arrayof pixels of intensity values. As shown in FIG. 8, the DWT isrecursively defined on one-dimensional data as a pair offilter-and-downsample operations. The highpass operation extracts smallscale information from the signal. The lowpass operation eliminatessmall scale content before downsampling. As further shown in FIG. 8, thelowpass image can undergo further filter-and-downsample operations,resulting in a “ladder” of information at multiple scales. With eachstep down the ladder, the frequency content represented is one octavelower, and the sampling rate is one half that of the next higher level,thus leading also in this case to an O(n) complexity. Exemplary filtercoefficients (LP) for the lowpass modules are LP=[⅛ ½ ¾ ½ ⅛]. Exemplaryfilter coefficients (HP) for the highpass modules are HP=[−½ 1 −½]. Thevarious operations can be easily performed with bit-shifts and adds,giving raise to the wavelet shown at the bottom of FIG. 8.

FIG. 9 shows application of the DWT in a two-dimensional environment. Alowpass/downsample (410) and highpass/downsample (420) first leveloperation is initially done in a horizontal direction. The outputs ofboth the horizontal highpass and horizontal downpass block are theninput to two separate vertical lowpass/downsample blocks (430), (440)and to two separate vertical highpass/downsample blocks (450), (460).The person skilled in the art will understand that the initial operationcan also be done in a vertical/horizontal fashion instead ofhorizontal/vertical as shown. This two-dimensional extension of the DWTimplicitly does orientation filtering, so that post-filtering steps areunnecessary.

The approach described above divides the image into four componentswhich contain information from vertical edges, horizontal edges,diagonal edges, and a lowpass intensity approximation (470). Suchapproximation is sent to a second level (480) having ahorizontal/vertical (or vertical/horizontal) configuration identical tothe first level, and iteration is performed, similarly to theone-dimensional environment.

After a DWT of the image is done, the result is rectified, to eliminateoscillations and show the power present at each combination of locationand scale. Data from each scale are then convolved with adifference-of-gaussians filter which deemphasizes areas of consistentspectral content (thus filtering the maps to suppress areas of similarenergy) while emphasizing areas which stand out from their surroundings.Finally, the inverse wavelet transform is taken with a non-oscillatoryfilter to normalize the maps. A lowpass filter is used to reconstructboth the lowpass and highpass components. In this way, oscillatoryfeatures are replaced with a general elevation of the signal, showingareas where high frequency spectral content is localized. The salientlocations will be identified as local maxima of the output signal. Inparticular, the local maxima will be located and sorted based on theirproximity to other maxima.

Use of DWT in place of the oriented Laplacian pyramid of Itti et al andelimination of complex-valued filtering gives an approximate 4× increasein speed over Itti et al. Optimizations in other filtering operationscan provide additional speedups. For example, a 61×61 image can beprocessed in about 50,000 operations, making the method viable forreal-time implementation on a processor such as a DSP.

According to a further embodiment of the present disclosure, videoinformation having a first resolution is processed to information havinga second, lower, resolution by way of a saliency method, and then sentto a visual stimulator on a patient. The saliency method will also allow“peripheral” information of an image to be processed by a patient oncesuch information is salient, and not only “central” information of theimage (which would be the usual outcome of a higher-to-lower resolutionprocessing). According to a yet further embodiment, the way of sendingsuch information to the patient can include selective activation ofelectrodes (e.g., edge electrodes only) or even activation of some“cues”, such as a tactile cue or an auditory cue. Therefore, accordingto this embodiment, saliency is used to cue different regions in theperipheral visual field.

More specifically, the field of view of a camera adapted to be used witha visual prosthesis is greater than the field of view of the prosthesis(20 degrees). Thus only the central 20 degrees of information from eachimage/video frame would be stimulated whereas the peripheral informationfalling outside the 20 degrees central field of view will be lost. Onthe other hand, the saliency module according to the present disclosurewill take in the entire video/image frame and process it for salientregions. If there are salient regions falling outside the central 20degree field of view, the patient could be alerted to the presence ofsalient regions using video/audio/tactile cues. The cues will be in oneof the 8 directions: right, left, top, bottom, top-left, bottom-left,top-right and bottom-right with respect to the central field of view.Thus recipients can move their head and divert their attention toimportant things in the periphery. In particular, audio cues would utterthe directions of the cues, video cues would mean stimulating peripheralelectrodes corresponding to the cueing directions.

According to another embodiment of the present disclosure, instead ofrunning continuously on the DSP at a certain frame rate, the saliencyalgorithm could be used to give cues to subjects when they demand them(saliency “on demand”). This could help patient familiarize themselveswith unknown environments etc and to know the area around them when theywish to do so. Running saliency on a continuous basis might putchallenging computational loads on the DSP and at the same time might betoo confusing for patients because in that case, they would becontinuously receiving cues which could make things more complicated forthem to comprehend.

According to a yet further embodiment of the present disclosure,saliency is applied to multiple images. In this environment, a movingobject is almost always more salient than a stationary object, so thatone could also cue on motion. In fact, that is how natural vision works:the peripheral vision is adapted to cue on motion. One sees motion “inthe corner of her eye” and she naturally looks at it. In the same way,motion cue can be combined with the “peripheral” embodiment to obtain asystem that acts more like natural vision.

In accordance with a further embodiment of the disclosure, patternrecognition can also be added as a feature. In particular, a table ofsalient stored patterns (e.g., a chair or a car) is stored in the visualprosthesis and compared with the salient features identified by thesaliency module, so that each time there is a match the patient isalerted to that pattern.

In summary, according to some embodiments of the present disclosure,saliency-based apparatus and methods for visual prostheses aredisclosed. A saliency-based component processes video data output by adigital signal processor before the video data are input to the retinalstimulator. In a saliency-based method, an intensity stream is extractedfrom an input image, feature maps based on the intensity stream aredeveloped, plural most salient regions of the input image are detectedand one of the regions is selected as a highest saliency region.

Accordingly, what has been shown are saliency-based methods in visualprostheses. While the methodology has been described by means ofspecific embodiments and applications thereof, it is understood thatnumerous modifications and variations could be made thereto by thoseskilled in the art without departing from the spirit and scope of thedisclosure. It is therefore to be understood that within the scope ofthe claims, the disclosure may be practiced otherwise than asspecifically described herein.

What is claimed is:
 1. A method of providing video data to a subjectwith an implanted visual stimulator, the implanted visual stimulatorassociated to an image processor processing video data from an inputimage, comprising: extracting only an intensity stream from the inputimage; developing feature maps based on the intensity stream; detectinga plurality of most salient regions of the input image based on thefeature maps; selecting one salient region of the most salient regionsas a highest saliency region; and providing highest saliency regioninformation to the subject through the implanted visual stimulator. 2.The method of claim 1, wherein the plurality of most salient regions arethree most salient regions.
 3. The method of claim 1, further comprisingconstructing an image pyramid for the intensity stream after extractingonly an intensity stream from the input image and before developingfeature maps based on the intensity stream.
 4. The method of claim 1,further comprising constructing conspicuity maps after developingfeature maps based on the intensity stream and before detecting aplurality of most salient regions of the input image based on thefeature maps.
 5. The method of claim 1, further comprising averagingeach region of the plurality of most salient regions after detecting theplurality of most salient regions and before selecting one salientregion of the most salient regions as a highest saliency region.
 6. Themethod of claim 1, the method being performed in absence of video datanormalization.
 7. The method of claim 1, wherein also information in theinfrared, ultraviolet, or X-ray spectra, or ultrasound information isprovided.
 8. The method of claim 1, wherein the method is applied to theinput image inside and outside the field of view of the subject.
 9. Themethod of claim 1, further comprising separating the input image intomultiple regions based on saliency.
 10. The method of claim 9, whereinseparating the input image into multiple regions generates high saliencyregions, medium saliency regions and low saliency regions.
 11. Themethod of claim 10, wherein the low saliency regions are groupedtogether in time during stimulation, while the high saliency regions andthe medium saliency regions are administered independently of eachother.
 12. The method of claim 10, wherein the high saliency regions areadministered before the medium saliency regions and the medium saliencyregions are administered before the low saliency regions.
 13. The methodof claim 34, wherein regions of the input image falling below a saliencythreshold are not presented to the subject.